{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feature selection is the process of selecting a subset of relevant features (variables, predictors) for use in model construction. Feature selection techniques are used for several reasons: to simplify models to make them easier to interpret, to reduce training time, to avoid the curse of dimensionality, to improve generalization by reducing overfitting (formally, variance reduction), and others.\n",
    "\n",
    "Skforecast is compatible with the **feature selection methods** implemented in [scikit-learn](https://scikit-learn.org/stable/modules/feature_selection.html) and [feature-engine](https://feature-engine.trainindata.com/en/latest/api_doc/selection/index.html) libraries. There are several methods for feature selection, but the most common are: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Recursive feature elimination**\n",
    "\n",
    "Given an external estimator that assigns weights to features (e.g., the coefficients of a linear model), the goal of recursive feature elimination ([RFE](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFE.html)) is to select features by recursively considering smaller and smaller sets of features. First, the estimator is trained on the initial set of features, and the importance of each feature is obtained either by a specific attribute (such as `coef_`, `feature_importances_`) or by a `callable`. Then, the least important features are pruned from the current set of features. This procedure is repeated recursively on the pruned set until the desired number of features to select is eventually reached. [`RFECV`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFECV.html#sklearn.feature_selection.RFECV) performs RFE in a cross-validation loop to find the optimal number of features.\n",
    "\n",
    "**Sequential Feature Selection**\n",
    "\n",
    "Sequential Feature Selection ([`SFS`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SequentialFeatureSelector.html)) can be either forward or backward, with the `direction` parameter controlling whether forward or backward SFS is used.\n",
    "\n",
    "+ **Forward-SFS** is a greedy procedure that iteratively finds the best new feature to add to the set of selected features. It starts with zero features and finds the one that maximizes a cross-validated score when an estimator is trained on that single feature. Once this first feature is selected, the procedure is repeated, adding one new feature to the set of selected features. The procedure stops when the desired number of selected features is reached, as determined by the `n_features_to_select` parameter.\n",
    "\n",
    "+ **Backward-SFS** follows the same idea, but works in the opposite direction. Instead of starting with no features and greedily adding features, it starts with all features and greedily removes features from the set. \n",
    "\n",
    "In general, forward and backward selection do not produce equivalent results. Also, one can be much faster than the other depending on the requested number of selected features: if we have 10 features and ask for 7 selected features, forward selection would need to perform 7 iterations while backward selection would only need to perform 3.\n",
    "\n",
    "[`SFS`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SequentialFeatureSelector.html) differs does not require the underlying model to expose a `coef_` or `feature_importances_` attribute. However, it may be slower compared to the other approaches, considering that more models have to be evaluated. For example in backward selection, the iteration going from $m$ features to $m - 1$ features using k-fold cross-validation requires fitting $m * k$ models to be evaluated.\n",
    "\n",
    "**Feature selection based on threshold (SelectFromModel)**\n",
    "\n",
    "[`SelectFromModel`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html) can be used along with any estimator that has a `coef_` or `feature_importances_` attribute after fitting. Features are considered unimportant and removed, if the corresponding `coef_` or `feature_importances_` values are below the given `threshold` parameter. In addition to specifying the `threshold` numerically, there are built-in heuristics for finding a threshold using a string argument. Available heuristics are `'mean'`, `'median'` and float multiples of these, such as `'0.1*mean'`.\n",
    "\n",
    "This method is very fast compared to the others because it does not require any additional model training. However, it does not evaluate the impact of feature removal on the model. It is often used for an initial selection before applying another more computationally expensive feature selection method.\n",
    "\n",
    "**Minimum Redundancy Maximum Relevance (MRMR)**\n",
    "\n",
    "Minimum Redundancy Maximum Relevance (MRMR) is a filter-based feature selection method that aims to identify a subset of features that are both highly relevant to the target variable and minimally redundant with respect to each other. Relevance is typically measured using mutual information between each feature and the target, while redundancy is assessed via the mutual information between pairs of features. By optimizing both criteria, mRMR helps reduce overfitting and improve model interpretability, especially in high-dimensional settings. The [`MRMR`](https://feature-engine.trainindata.com/en/latest/user_guide/selection/MRMR.html) class from [feature-engine](https://feature-engine.trainindata.com/en/latest/index.html) can be used to implement this method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,191,191,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00bfa5; border-color: #00bfa5; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00bfa5;\"></i>\n",
    "    <b style=\"color: #00bfa5;\">&#128161 Tip</b>\n",
    "</p>\n",
    "\n",
    "Feature selection is a powerful tool for improving the performance of machine learning models. However, it is computationally expensive and can be time-consuming. Since the goal is to find the best subset of features, not the best model, it is not necessary to use the entire data set or a highly complex model. Instead, it is recommended to use a <b>small subset of the data and a simple model</b>. Once the best subset of features has been identified, the model can then be trained using the entire dataset and a more complex configuration.\n",
    "<br><br>\n",
    "For example, in this use case, the model is an <code>LGMBRegressor</code> with 900 trees and a maximum depth of 7. However, to find the best subset of features, only 100 trees and a maximum depth of 5 are used.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature selection with skforecast"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `select_features` and `select_features_multiseries` functions can be used to select the best subset of features (autoregressive and exogenous variables). These functions are compatible with the feature selection methods implemented in the scikit-learn library. The available parameters are:\n",
    "\n",
    "- `forecaster`: Forecaster of type `ForecasterRecursive`,  `ForecasterDirect`, `ForecasterRecursiveMultiSeries` or `ForecasterDirectMultiVariate`.\n",
    "\n",
    "- `selector`: Feature selector from `sklearn.feature_selection`. For example, `RFE` or `RFECV`.\n",
    "\n",
    "- `y` or `series`: Target time series to which the feature selection will be applied.\n",
    "\n",
    "- `exog`: Exogenous variables.\n",
    "\n",
    "- `select_only`: Decide what type of features to include in the selection process. \n",
    "        \n",
    "    + If `'autoreg'`, only autoregressive features (lags and window features) are evaluated by the selector. All exogenous features are included in the output `selected_exog`.\n",
    "\n",
    "    + If `'exog'`, only exogenous features are evaluated without the presence of autoregressive features. All autoregressive features are included in the outputs `selected_lags` and `selected_window_features`.\n",
    "\n",
    "    + If `None`, all features are evaluated by the selector.\n",
    "\n",
    "- `force_inclusion`: Features to force include in the final list of selected features.\n",
    "        \n",
    "    + If `list`, list of feature names to force include.\n",
    "    \n",
    "    + If `str`, regular expression to identify features to force include. For example, if `force_inclusion=\"^sun_\"`, all features that begin with \"sun_\" will be included in the final list of selected features.\n",
    "\n",
    "- `subsample`: Proportion of records to use for feature selection.\n",
    "\n",
    "- `random_state`: Sets a seed for the random subsample so that the subsampling process is always deterministic.\n",
    "\n",
    "- `verbose`: Print information about feature selection process.\n",
    "\n",
    "These functions return three `list`:\n",
    "\n",
    "- `selected_lags`: List of selected lags.\n",
    "\n",
    "- `selected_window_features`: List of selected window features.\n",
    "\n",
    "- `selected_exog`: List of selected exogenous features."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries and data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Libraries\n",
    "# ==============================================================================\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from lightgbm import LGBMRegressor\n",
    "from sklearn.feature_selection import RFECV\n",
    "from sklearn.feature_selection import SequentialFeatureSelector\n",
    "from sklearn.feature_selection import SelectFromModel\n",
    "from sklearn.model_selection import ShuffleSplit\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from feature_engine.selection import MRMR\n",
    "\n",
    "from skforecast.datasets import fetch_dataset\n",
    "from skforecast.preprocessing import RollingFeatures\n",
    "from skforecast.recursive import ForecasterRecursive\n",
    "from skforecast.recursive import ForecasterRecursiveMultiSeries\n",
    "from skforecast.feature_selection import select_features\n",
    "from skforecast.feature_selection import select_features_multiseries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">╭──────────────────────── <span style=\"font-weight: bold\">bike_sharing_extended_features</span> ─────────────────────────╮\n",
       "│ <span style=\"font-weight: bold\">Description:</span>                                                                    │\n",
       "│ Hourly usage of the bike share system in the city of Washington D.C. during the │\n",
       "│ years 2011 and 2012. In addition to the number of users per hour, the dataset   │\n",
       "│ was enriched by introducing supplementary features. Addition includes calendar- │\n",
       "│ based variables (day of the week, hour of the day, month, etc.), indicators for │\n",
       "│ sunlight, incorporation of rolling temperature averages, and the creation of    │\n",
       "│ polynomial features generated from variable pairs. All cyclic variables are     │\n",
       "│ encoded using sine and cosine functions to ensure accurate representation.      │\n",
       "│                                                                                 │\n",
       "│ <span style=\"font-weight: bold\">Source:</span>                                                                         │\n",
       "│ Fanaee-T,Hadi. (2013). Bike Sharing Dataset. UCI Machine Learning Repository.   │\n",
       "│ https://doi.org/10.24432/C5W894.                                                │\n",
       "│                                                                                 │\n",
       "│ <span style=\"font-weight: bold\">URL:</span>                                                                            │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-                        │\n",
       "│ datasets/main/data/bike_sharing_extended_features.csv                           │\n",
       "│                                                                                 │\n",
       "│ <span style=\"font-weight: bold\">Shape:</span> 17352 rows x 90 columns                                                  │\n",
       "╰─────────────────────────────────────────────────────────────────────────────────╯\n",
       "</pre>\n"
      ],
      "text/plain": [
       "╭──────────────────────── \u001b[1mbike_sharing_extended_features\u001b[0m ─────────────────────────╮\n",
       "│ \u001b[1mDescription:\u001b[0m                                                                    │\n",
       "│ Hourly usage of the bike share system in the city of Washington D.C. during the │\n",
       "│ years 2011 and 2012. In addition to the number of users per hour, the dataset   │\n",
       "│ was enriched by introducing supplementary features. Addition includes calendar- │\n",
       "│ based variables (day of the week, hour of the day, month, etc.), indicators for │\n",
       "│ sunlight, incorporation of rolling temperature averages, and the creation of    │\n",
       "│ polynomial features generated from variable pairs. All cyclic variables are     │\n",
       "│ encoded using sine and cosine functions to ensure accurate representation.      │\n",
       "│                                                                                 │\n",
       "│ \u001b[1mSource:\u001b[0m                                                                         │\n",
       "│ Fanaee-T,Hadi. (2013). Bike Sharing Dataset. UCI Machine Learning Repository.   │\n",
       "│ https://doi.org/10.24432/C5W894.                                                │\n",
       "│                                                                                 │\n",
       "│ \u001b[1mURL:\u001b[0m                                                                            │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-                        │\n",
       "│ datasets/main/data/bike_sharing_extended_features.csv                           │\n",
       "│                                                                                 │\n",
       "│ \u001b[1mShape:\u001b[0m 17352 rows x 90 columns                                                  │\n",
       "╰─────────────────────────────────────────────────────────────────────────────────╯\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>users</th>\n",
       "      <th>weather</th>\n",
       "      <th>month_sin</th>\n",
       "      <th>month_cos</th>\n",
       "      <th>week_of_year_sin</th>\n",
       "      <th>week_of_year_cos</th>\n",
       "      <th>week_day_sin</th>\n",
       "      <th>week_day_cos</th>\n",
       "      <th>hour_day_sin</th>\n",
       "      <th>hour_day_cos</th>\n",
       "      <th>...</th>\n",
       "      <th>temp_roll_mean_1_day</th>\n",
       "      <th>temp_roll_mean_7_day</th>\n",
       "      <th>temp_roll_max_1_day</th>\n",
       "      <th>temp_roll_min_1_day</th>\n",
       "      <th>temp_roll_max_7_day</th>\n",
       "      <th>temp_roll_min_7_day</th>\n",
       "      <th>holiday_previous_day</th>\n",
       "      <th>holiday_next_day</th>\n",
       "      <th>temp</th>\n",
       "      <th>holiday</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-08 00:00:00</th>\n",
       "      <td>25.0</td>\n",
       "      <td>mist</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.866025</td>\n",
       "      <td>0.120537</td>\n",
       "      <td>0.992709</td>\n",
       "      <td>-0.781832</td>\n",
       "      <td>0.62349</td>\n",
       "      <td>0.258819</td>\n",
       "      <td>0.965926</td>\n",
       "      <td>...</td>\n",
       "      <td>8.063334</td>\n",
       "      <td>10.127976</td>\n",
       "      <td>9.02</td>\n",
       "      <td>6.56</td>\n",
       "      <td>18.86</td>\n",
       "      <td>4.92</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.38</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-08 01:00:00</th>\n",
       "      <td>16.0</td>\n",
       "      <td>mist</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.866025</td>\n",
       "      <td>0.120537</td>\n",
       "      <td>0.992709</td>\n",
       "      <td>-0.781832</td>\n",
       "      <td>0.62349</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.866025</td>\n",
       "      <td>...</td>\n",
       "      <td>8.029166</td>\n",
       "      <td>10.113334</td>\n",
       "      <td>9.02</td>\n",
       "      <td>6.56</td>\n",
       "      <td>18.86</td>\n",
       "      <td>4.92</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.38</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-08 02:00:00</th>\n",
       "      <td>16.0</td>\n",
       "      <td>mist</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.866025</td>\n",
       "      <td>0.120537</td>\n",
       "      <td>0.992709</td>\n",
       "      <td>-0.781832</td>\n",
       "      <td>0.62349</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>...</td>\n",
       "      <td>7.995000</td>\n",
       "      <td>10.103572</td>\n",
       "      <td>9.02</td>\n",
       "      <td>6.56</td>\n",
       "      <td>18.86</td>\n",
       "      <td>4.92</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.38</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 90 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     users weather  month_sin  month_cos  week_of_year_sin  \\\n",
       "date_time                                                                    \n",
       "2011-01-08 00:00:00   25.0    mist        0.5   0.866025          0.120537   \n",
       "2011-01-08 01:00:00   16.0    mist        0.5   0.866025          0.120537   \n",
       "2011-01-08 02:00:00   16.0    mist        0.5   0.866025          0.120537   \n",
       "\n",
       "                     week_of_year_cos  week_day_sin  week_day_cos  \\\n",
       "date_time                                                           \n",
       "2011-01-08 00:00:00          0.992709     -0.781832       0.62349   \n",
       "2011-01-08 01:00:00          0.992709     -0.781832       0.62349   \n",
       "2011-01-08 02:00:00          0.992709     -0.781832       0.62349   \n",
       "\n",
       "                     hour_day_sin  hour_day_cos  ...  temp_roll_mean_1_day  \\\n",
       "date_time                                        ...                         \n",
       "2011-01-08 00:00:00      0.258819      0.965926  ...              8.063334   \n",
       "2011-01-08 01:00:00      0.500000      0.866025  ...              8.029166   \n",
       "2011-01-08 02:00:00      0.707107      0.707107  ...              7.995000   \n",
       "\n",
       "                     temp_roll_mean_7_day  temp_roll_max_1_day  \\\n",
       "date_time                                                        \n",
       "2011-01-08 00:00:00             10.127976                 9.02   \n",
       "2011-01-08 01:00:00             10.113334                 9.02   \n",
       "2011-01-08 02:00:00             10.103572                 9.02   \n",
       "\n",
       "                     temp_roll_min_1_day  temp_roll_max_7_day  \\\n",
       "date_time                                                       \n",
       "2011-01-08 00:00:00                 6.56                18.86   \n",
       "2011-01-08 01:00:00                 6.56                18.86   \n",
       "2011-01-08 02:00:00                 6.56                18.86   \n",
       "\n",
       "                     temp_roll_min_7_day  holiday_previous_day  \\\n",
       "date_time                                                        \n",
       "2011-01-08 00:00:00                 4.92                   0.0   \n",
       "2011-01-08 01:00:00                 4.92                   0.0   \n",
       "2011-01-08 02:00:00                 4.92                   0.0   \n",
       "\n",
       "                     holiday_next_day  temp  holiday  \n",
       "date_time                                             \n",
       "2011-01-08 00:00:00               0.0  7.38      0.0  \n",
       "2011-01-08 01:00:00               0.0  7.38      0.0  \n",
       "2011-01-08 02:00:00               0.0  7.38      0.0  \n",
       "\n",
       "[3 rows x 90 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Download data\n",
    "# ==============================================================================\n",
    "data = fetch_dataset(name=\"bike_sharing_extended_features\")\n",
    "data.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data selection (reduce data size to speed up the example)\n",
    "# ==============================================================================\n",
    "data = data.drop(columns=\"weather\")\n",
    "data = data.loc[\"2012-01-01 00:00:00\":]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create forecaster\n",
    "\n",
    "A forecasting model is created to predict the number of users using the last 48 values (last two days) and the exogenous features available in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create forecaster\n",
    "# ==============================================================================\n",
    "window_features = RollingFeatures(\n",
    "                      stats        = ['mean', 'mean', 'sum'],\n",
    "                      window_sizes = [24, 48, 24]\n",
    "                  )\n",
    "\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator       = LGBMRegressor(\n",
    "                                       n_estimators = 900,\n",
    "                                       random_state = 15926,\n",
    "                                       max_depth    = 7,\n",
    "                                       verbose      = -1\n",
    "                                   ),\n",
    "                 lags            = 48,\n",
    "                 window_features = window_features\n",
    "             )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature selection with Recursive Feature Elimination (RFECV)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selection of autoregressive and exogenous features\n",
    "\n",
    "By default, the `select_features` function selects the best subset of autoregressive and exogenous features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (RFECV)\n",
      "-------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 4356\n",
      "Number of features available: 139\n",
      "    Lags            (n=48)\n",
      "    Window features (n=3)\n",
      "    Exog            (n=88)\n",
      "Number of features selected: 58\n",
      "    Lags            (n=36) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 19, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 40, 41, 42, 44, 46, 47, 48]\n",
      "    Window features (n=1) : ['roll_mean_24']\n",
      "    Exog            (n=21) : ['hour_day_sin', 'hour_day_cos', 'poly_month_cos__week_of_year_sin', 'poly_month_cos__hour_day_cos', 'poly_week_of_year_sin__week_day_sin', 'poly_week_of_year_sin__week_day_cos', 'poly_week_of_year_sin__hour_day_sin', 'poly_week_of_year_sin__hour_day_cos', 'poly_week_of_year_sin__sunset_hour_cos', 'poly_week_of_year_cos__week_day_sin', 'poly_week_of_year_cos__week_day_cos', 'poly_week_day_sin__hour_day_sin', 'poly_week_day_sin__hour_day_cos', 'poly_week_day_cos__hour_day_sin', 'poly_week_day_cos__hour_day_cos', 'poly_week_day_cos__sunset_hour_sin', 'poly_hour_day_sin__hour_day_cos', 'poly_hour_day_cos__sunset_hour_sin', 'temp_roll_mean_1_day', 'temp_roll_mean_7_day', 'temp']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (autoregressive and exog) with scikit-learn RFECV\n",
    "# ==============================================================================\n",
    "import warnings\n",
    "warnings.filterwarnings(\n",
    "    \"ignore\",\n",
    "    message=\"X does not have valid feature names, but .* was fitted with feature names\"\n",
    ")\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "selector = RFECV(estimator=estimator, step=1, cv=3, min_features_to_select=25)\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster      = forecaster,\n",
    "    selector        = selector,\n",
    "    y               = data[\"users\"],\n",
    "    exog            = data.drop(columns=\"users\"),\n",
    "    select_only     = None,\n",
    "    force_inclusion = None,\n",
    "    subsample       = 0.5,\n",
    "    random_state    = 123,\n",
    "    verbose         = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, the Forecaster model is trained with the selected features. As the window features are generated with the `RollingFeatures` class, the selected window features must be included manually creating a new object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train forecaster with selected features\n",
    "# ==============================================================================\n",
    "new_window_features = RollingFeatures(\n",
    "                          stats        = ['mean'],\n",
    "                          window_sizes = 24\n",
    "                      )\n",
    "\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator       = LGBMRegressor(\n",
    "                                       n_estimators = 900,\n",
    "                                       random_state = 15926,\n",
    "                                       max_depth    = 7,\n",
    "                                       verbose      = -1\n",
    "                                   ),\n",
    "                 lags            = selected_lags,\n",
    "                 window_features = new_window_features\n",
    "             )\n",
    "\n",
    "forecaster.fit(y=data[\"users\"], exog=data[selected_exog])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selection on a subset of features\n",
    "\n",
    "+ If `select_only = 'autoreg'`, only autoregressive features (lags or custom predictors) are evaluated by the selector. All exogenous features are included in the output `selected_exog`.\n",
    "\n",
    "+ If `select_only = 'exog'`, exogenous features are evaluated by the selector in the absence of autoregressive features. All autoregressive features are included in the outputs `selected_lags` and `selected_window_features`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (RFECV)\n",
      "-------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 4356\n",
      "Number of features available: 125\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=88)\n",
      "Number of features selected: 33\n",
      "    Lags            (n=33) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 19, 22, 23, 24, 25, 26, 28, 29, 30, 32, 34, 35, 36, 40, 41, 42, 44, 46, 48]\n",
      "    Window features (n=0) : []\n",
      "    Exog            (n=88) : ['month_sin', 'month_cos', 'week_of_year_sin', 'week_of_year_cos', 'week_day_sin', 'week_day_cos', 'hour_day_sin', 'hour_day_cos', 'sunrise_hour_sin', 'sunrise_hour_cos', 'sunset_hour_sin', 'sunset_hour_cos', 'poly_month_sin__month_cos', 'poly_month_sin__week_of_year_sin', 'poly_month_sin__week_of_year_cos', 'poly_month_sin__week_day_sin', 'poly_month_sin__week_day_cos', 'poly_month_sin__hour_day_sin', 'poly_month_sin__hour_day_cos', 'poly_month_sin__sunrise_hour_sin', 'poly_month_sin__sunrise_hour_cos', 'poly_month_sin__sunset_hour_sin', 'poly_month_sin__sunset_hour_cos', 'poly_month_cos__week_of_year_sin', 'poly_month_cos__week_of_year_cos', 'poly_month_cos__week_day_sin', 'poly_month_cos__week_day_cos', 'poly_month_cos__hour_day_sin', 'poly_month_cos__hour_day_cos', 'poly_month_cos__sunrise_hour_sin', 'poly_month_cos__sunrise_hour_cos', 'poly_month_cos__sunset_hour_sin', 'poly_month_cos__sunset_hour_cos', 'poly_week_of_year_sin__week_of_year_cos', 'poly_week_of_year_sin__week_day_sin', 'poly_week_of_year_sin__week_day_cos', 'poly_week_of_year_sin__hour_day_sin', 'poly_week_of_year_sin__hour_day_cos', 'poly_week_of_year_sin__sunrise_hour_sin', 'poly_week_of_year_sin__sunrise_hour_cos', 'poly_week_of_year_sin__sunset_hour_sin', 'poly_week_of_year_sin__sunset_hour_cos', 'poly_week_of_year_cos__week_day_sin', 'poly_week_of_year_cos__week_day_cos', 'poly_week_of_year_cos__hour_day_sin', 'poly_week_of_year_cos__hour_day_cos', 'poly_week_of_year_cos__sunrise_hour_sin', 'poly_week_of_year_cos__sunrise_hour_cos', 'poly_week_of_year_cos__sunset_hour_sin', 'poly_week_of_year_cos__sunset_hour_cos', 'poly_week_day_sin__week_day_cos', 'poly_week_day_sin__hour_day_sin', 'poly_week_day_sin__hour_day_cos', 'poly_week_day_sin__sunrise_hour_sin', 'poly_week_day_sin__sunrise_hour_cos', 'poly_week_day_sin__sunset_hour_sin', 'poly_week_day_sin__sunset_hour_cos', 'poly_week_day_cos__hour_day_sin', 'poly_week_day_cos__hour_day_cos', 'poly_week_day_cos__sunrise_hour_sin', 'poly_week_day_cos__sunrise_hour_cos', 'poly_week_day_cos__sunset_hour_sin', 'poly_week_day_cos__sunset_hour_cos', 'poly_hour_day_sin__hour_day_cos', 'poly_hour_day_sin__sunrise_hour_sin', 'poly_hour_day_sin__sunrise_hour_cos', 'poly_hour_day_sin__sunset_hour_sin', 'poly_hour_day_sin__sunset_hour_cos', 'poly_hour_day_cos__sunrise_hour_sin', 'poly_hour_day_cos__sunrise_hour_cos', 'poly_hour_day_cos__sunset_hour_sin', 'poly_hour_day_cos__sunset_hour_cos', 'poly_sunrise_hour_sin__sunrise_hour_cos', 'poly_sunrise_hour_sin__sunset_hour_sin', 'poly_sunrise_hour_sin__sunset_hour_cos', 'poly_sunrise_hour_cos__sunset_hour_sin', 'poly_sunrise_hour_cos__sunset_hour_cos', 'poly_sunset_hour_sin__sunset_hour_cos', 'temp_roll_mean_1_day', 'temp_roll_mean_7_day', 'temp_roll_max_1_day', 'temp_roll_min_1_day', 'temp_roll_max_7_day', 'temp_roll_min_7_day', 'holiday_previous_day', 'holiday_next_day', 'temp', 'holiday']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (only autoregressive) with scikit-learn RFECV\n",
    "# ==============================================================================\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "selector = RFECV(estimator=estimator, step=1, cv=3, min_features_to_select=25)\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster  = forecaster,\n",
    "    selector    = selector,\n",
    "    y           = data[\"users\"],\n",
    "    exog        = data.drop(columns=\"users\"),\n",
    "    select_only = 'autoreg',\n",
    "    subsample   = 0.5,\n",
    "    verbose     = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check all exogenous features are selected\n",
    "# ==============================================================================\n",
    "len(selected_exog) == data.drop(columns=\"users\").shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (RFECV)\n",
      "-------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 4356\n",
      "Number of features available: 125\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=88)\n",
      "Number of features selected: 61\n",
      "    Lags            (n=36) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 19, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 40, 41, 42, 44, 46, 47, 48]\n",
      "    Window features (n=1) : ['roll_mean_24']\n",
      "    Exog            (n=61) : ['week_of_year_sin', 'week_of_year_cos', 'week_day_sin', 'week_day_cos', 'hour_day_sin', 'hour_day_cos', 'poly_month_sin__week_of_year_sin', 'poly_month_sin__week_of_year_cos', 'poly_month_sin__week_day_sin', 'poly_month_sin__week_day_cos', 'poly_month_sin__hour_day_sin', 'poly_month_sin__hour_day_cos', 'poly_month_cos__week_of_year_sin', 'poly_month_cos__week_day_sin', 'poly_month_cos__week_day_cos', 'poly_month_cos__hour_day_sin', 'poly_month_cos__hour_day_cos', 'poly_week_of_year_sin__week_of_year_cos', 'poly_week_of_year_sin__week_day_sin', 'poly_week_of_year_sin__week_day_cos', 'poly_week_of_year_sin__hour_day_sin', 'poly_week_of_year_sin__hour_day_cos', 'poly_week_of_year_sin__sunrise_hour_sin', 'poly_week_of_year_sin__sunrise_hour_cos', 'poly_week_of_year_sin__sunset_hour_sin', 'poly_week_of_year_sin__sunset_hour_cos', 'poly_week_of_year_cos__week_day_sin', 'poly_week_of_year_cos__week_day_cos', 'poly_week_of_year_cos__hour_day_sin', 'poly_week_of_year_cos__hour_day_cos', 'poly_week_of_year_cos__sunrise_hour_cos', 'poly_week_of_year_cos__sunset_hour_sin', 'poly_week_day_sin__week_day_cos', 'poly_week_day_sin__hour_day_sin', 'poly_week_day_sin__hour_day_cos', 'poly_week_day_sin__sunrise_hour_sin', 'poly_week_day_sin__sunrise_hour_cos', 'poly_week_day_sin__sunset_hour_sin', 'poly_week_day_sin__sunset_hour_cos', 'poly_week_day_cos__hour_day_sin', 'poly_week_day_cos__hour_day_cos', 'poly_week_day_cos__sunrise_hour_sin', 'poly_week_day_cos__sunrise_hour_cos', 'poly_week_day_cos__sunset_hour_sin', 'poly_week_day_cos__sunset_hour_cos', 'poly_hour_day_sin__hour_day_cos', 'poly_hour_day_sin__sunrise_hour_sin', 'poly_hour_day_sin__sunrise_hour_cos', 'poly_hour_day_sin__sunset_hour_sin', 'poly_hour_day_sin__sunset_hour_cos', 'poly_hour_day_cos__sunrise_hour_sin', 'poly_hour_day_cos__sunrise_hour_cos', 'poly_hour_day_cos__sunset_hour_sin', 'temp_roll_mean_1_day', 'temp_roll_mean_7_day', 'temp_roll_max_1_day', 'temp_roll_min_1_day', 'temp_roll_max_7_day', 'temp_roll_min_7_day', 'temp', 'holiday']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (only exog) with scikit-learn RFECV\n",
    "# ==============================================================================\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "selector = RFECV(estimator=estimator, step=1, cv=3, min_features_to_select=25)\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster  = forecaster,\n",
    "    selector    = selector,\n",
    "    y           = data[\"users\"],\n",
    "    exog        = data.drop(columns=\"users\"),\n",
    "    select_only = 'exog',\n",
    "    subsample   = 0.5,\n",
    "    verbose     = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Same lags : True\n",
      "Same window features : True\n"
     ]
    }
   ],
   "source": [
    "# Check all autoregressive features are selected\n",
    "# ==============================================================================\n",
    "print(\"Same lags :\", len(selected_lags) == len(forecaster.lags))\n",
    "print(\"Same window features :\", len(selected_window_features) == len(forecaster.window_features))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Force selection of specific features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `force_inclusion` argument can be used to force the selection of certain features. To illustrate this, a non-informative feature is added to the data set, `noise`. This feature contains no information about the target variable and therefore should not be selected by the feature selector. However, if we force the inclusion of this feature, it will be included in the final list of selected features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add non-informative feature\n",
    "# ==============================================================================\n",
    "data['noise'] = np.random.normal(size=len(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (RFECV)\n",
      "-------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 4356\n",
      "Number of features available: 126\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=89)\n",
      "Number of features selected: 80\n",
      "    Lags            (n=36) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 19, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 40, 41, 42, 44, 46, 47, 48]\n",
      "    Window features (n=1) : ['roll_mean_24']\n",
      "    Exog            (n=80) : ['month_sin', 'month_cos', 'week_of_year_sin', 'week_of_year_cos', 'week_day_sin', 'week_day_cos', 'hour_day_sin', 'hour_day_cos', 'sunrise_hour_sin', 'poly_month_sin__month_cos', 'poly_month_sin__week_of_year_sin', 'poly_month_sin__week_of_year_cos', 'poly_month_sin__week_day_sin', 'poly_month_sin__week_day_cos', 'poly_month_sin__hour_day_sin', 'poly_month_sin__hour_day_cos', 'poly_month_sin__sunrise_hour_cos', 'poly_month_sin__sunset_hour_sin', 'poly_month_sin__sunset_hour_cos', 'poly_month_cos__week_of_year_sin', 'poly_month_cos__week_of_year_cos', 'poly_month_cos__week_day_sin', 'poly_month_cos__week_day_cos', 'poly_month_cos__hour_day_sin', 'poly_month_cos__hour_day_cos', 'poly_month_cos__sunrise_hour_sin', 'poly_month_cos__sunset_hour_cos', 'poly_week_of_year_sin__week_of_year_cos', 'poly_week_of_year_sin__week_day_sin', 'poly_week_of_year_sin__week_day_cos', 'poly_week_of_year_sin__hour_day_sin', 'poly_week_of_year_sin__hour_day_cos', 'poly_week_of_year_sin__sunrise_hour_sin', 'poly_week_of_year_sin__sunrise_hour_cos', 'poly_week_of_year_sin__sunset_hour_sin', 'poly_week_of_year_sin__sunset_hour_cos', 'poly_week_of_year_cos__week_day_sin', 'poly_week_of_year_cos__week_day_cos', 'poly_week_of_year_cos__hour_day_sin', 'poly_week_of_year_cos__hour_day_cos', 'poly_week_of_year_cos__sunrise_hour_sin', 'poly_week_of_year_cos__sunrise_hour_cos', 'poly_week_of_year_cos__sunset_hour_sin', 'poly_week_of_year_cos__sunset_hour_cos', 'poly_week_day_sin__week_day_cos', 'poly_week_day_sin__hour_day_sin', 'poly_week_day_sin__hour_day_cos', 'poly_week_day_sin__sunrise_hour_sin', 'poly_week_day_sin__sunrise_hour_cos', 'poly_week_day_sin__sunset_hour_sin', 'poly_week_day_sin__sunset_hour_cos', 'poly_week_day_cos__hour_day_sin', 'poly_week_day_cos__hour_day_cos', 'poly_week_day_cos__sunrise_hour_sin', 'poly_week_day_cos__sunrise_hour_cos', 'poly_week_day_cos__sunset_hour_sin', 'poly_week_day_cos__sunset_hour_cos', 'poly_hour_day_sin__hour_day_cos', 'poly_hour_day_sin__sunrise_hour_sin', 'poly_hour_day_sin__sunrise_hour_cos', 'poly_hour_day_sin__sunset_hour_sin', 'poly_hour_day_sin__sunset_hour_cos', 'poly_hour_day_cos__sunrise_hour_sin', 'poly_hour_day_cos__sunrise_hour_cos', 'poly_hour_day_cos__sunset_hour_sin', 'poly_hour_day_cos__sunset_hour_cos', 'poly_sunrise_hour_sin__sunset_hour_sin', 'poly_sunrise_hour_sin__sunset_hour_cos', 'poly_sunrise_hour_cos__sunset_hour_sin', 'temp_roll_mean_1_day', 'temp_roll_mean_7_day', 'temp_roll_max_1_day', 'temp_roll_min_1_day', 'temp_roll_max_7_day', 'temp_roll_min_7_day', 'holiday_previous_day', 'holiday_next_day', 'temp', 'holiday', 'noise']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (only exog) with scikit-learn RFECV\n",
    "# ==============================================================================\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "selector = RFECV(estimator=estimator, step=1, cv=3, min_features_to_select=10)\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster      = forecaster,\n",
    "    selector        = selector,\n",
    "    y               = data[\"users\"],\n",
    "    exog            = data.drop(columns=\"users\"),\n",
    "    select_only     = 'exog',\n",
    "    force_inclusion = [\"noise\"],\n",
    "    subsample       = 0.5,\n",
    "    verbose         = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check if \"noise\" is in selected_exog\n",
    "# ==============================================================================\n",
    "\"noise\" in selected_exog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature selection with Sequential Feature Selection (SFS)\n",
    "\n",
    "Sequential Feature Selection is a robust method for selecting features, but it is **computationally expensive**. When the data set is very large, one way to reduce the computational cost is to use a single validation split to evaluate each candidate model instead of cross-validation (default)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (SequentialFeatureSelector)\n",
      "---------------------------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 1742\n",
      "Number of features available: 126\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=89)\n",
      "Number of features selected: 25\n",
      "    Lags            (n=36) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 19, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 40, 41, 42, 44, 46, 47, 48]\n",
      "    Window features (n=1) : ['roll_mean_24']\n",
      "    Exog            (n=25) : ['week_of_year_sin', 'week_day_sin', 'hour_day_sin', 'hour_day_cos', 'sunset_hour_sin', 'poly_month_sin__week_day_cos', 'poly_month_cos__week_of_year_cos', 'poly_month_cos__sunset_hour_cos', 'poly_week_of_year_sin__sunset_hour_sin', 'poly_week_of_year_cos__sunrise_hour_cos', 'poly_week_of_year_cos__sunset_hour_cos', 'poly_week_day_sin__week_day_cos', 'poly_week_day_sin__sunrise_hour_sin', 'poly_week_day_sin__sunset_hour_sin', 'poly_week_day_cos__hour_day_cos', 'poly_hour_day_sin__hour_day_cos', 'poly_hour_day_cos__sunset_hour_sin', 'poly_sunrise_hour_sin__sunset_hour_sin', 'poly_sunrise_hour_cos__sunset_hour_sin', 'poly_sunrise_hour_cos__sunset_hour_cos', 'temp_roll_max_1_day', 'temp_roll_min_1_day', 'holiday_next_day', 'temp', 'holiday']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (only exog) with scikit-learn SequentialFeatureSelector\n",
    "# ==============================================================================\n",
    "estimator = LGBMRegressor(n_estimators=50, max_depth=3, random_state=15926, verbose=-1)\n",
    "selector = SequentialFeatureSelector(\n",
    "               estimator            = forecaster.estimator,\n",
    "               n_features_to_select = 25,\n",
    "               direction            = \"forward\",\n",
    "               cv                   = ShuffleSplit(n_splits=1, test_size=0.3, random_state=951),\n",
    "               scoring              = \"neg_mean_absolute_error\",\n",
    "           )\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster   = forecaster,\n",
    "    selector     = selector,\n",
    "    y            = data[\"users\"],\n",
    "    exog         = data.drop(columns=\"users\"),\n",
    "    select_only  = 'exog',\n",
    "    subsample    = 0.2,\n",
    "    random_state = 123,\n",
    "    verbose      = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature selection with Minimum Redundancy Maximum Relevance (MRMR)\n",
    "\n",
    "Minimum Redundancy Maximum Relevance (MRMR) is a filter-based feature selection method that is **model-agnostic and fast to compute**, making it suitable for high-dimensional datasets. However, it relies on statistical criteria like mutual information rather than evaluating model performance directly, so it may be less tailored to a specific estimator compared to wrapper methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\feature_engine\\selection\\mrmr.py:465: RuntimeWarning: divide by zero encountered in divide\n",
      "  mrmr = relevance / redundance\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (MRMR)\n",
      "------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 4356\n",
      "Number of features available: 126\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=89)\n",
      "Number of features selected: 25\n",
      "    Lags            (n=17) : [1, 2, 3, 4, 5, 11, 14, 16, 22, 23, 24, 25, 26, 34, 46, 47, 48]\n",
      "    Window features (n=0) : []\n",
      "    Exog            (n=8) : ['hour_day_cos', 'poly_month_sin__month_cos', 'poly_week_day_cos__hour_day_sin', 'poly_week_day_cos__sunrise_hour_cos', 'holiday_next_day', 'temp', 'holiday', 'noise']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection with feature-engine MRMR\n",
    "# ==============================================================================\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "selector = MRMR(method=\"MIQ\", max_features=25, regression=True, cv=3)\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster   = forecaster,\n",
    "    selector     = selector,\n",
    "    y            = data[\"users\"],\n",
    "    exog         = data.drop(columns=\"users\"),\n",
    "    select_only  = None,\n",
    "    subsample    = 0.5,\n",
    "    random_state = 123,\n",
    "    verbose      = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combination of feature selection methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combining feature selection methods can help speed up the process. An effective approach is to first use `SelectFromModel` to eliminate the less important features, and then use `SequentialFeatureSelector` to determine the best subset of features from this reduced list. This two-step method often improves efficiency by focusing on the most important features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (SelectFromModel)\n",
      "-----------------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 1742\n",
      "Number of features available: 126\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=89)\n",
      "Number of features selected: 62\n",
      "    Lags            (n=36) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 19, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 40, 41, 42, 44, 46, 47, 48]\n",
      "    Window features (n=1) : [np.str_('roll_mean_24')]\n",
      "    Exog            (n=25) : [np.str_('week_of_year_sin'), np.str_('hour_day_sin'), np.str_('hour_day_cos'), np.str_('poly_month_sin__week_day_sin'), np.str_('poly_month_sin__week_day_cos'), np.str_('poly_month_cos__week_of_year_sin'), np.str_('poly_week_of_year_sin__week_day_cos'), np.str_('poly_week_of_year_sin__hour_day_sin'), np.str_('poly_week_of_year_sin__hour_day_cos'), np.str_('poly_week_of_year_cos__week_day_sin'), np.str_('poly_week_of_year_cos__week_day_cos'), np.str_('poly_week_day_sin__hour_day_sin'), np.str_('poly_week_day_sin__hour_day_cos'), np.str_('poly_week_day_sin__sunrise_hour_sin'), np.str_('poly_week_day_sin__sunrise_hour_cos'), np.str_('poly_week_day_cos__hour_day_sin'), np.str_('poly_week_day_cos__hour_day_cos'), np.str_('poly_hour_day_sin__hour_day_cos'), np.str_('poly_hour_day_sin__sunrise_hour_cos'), np.str_('poly_hour_day_sin__sunset_hour_sin'), np.str_('poly_hour_day_cos__sunset_hour_sin'), np.str_('temp_roll_mean_7_day'), np.str_('temp_roll_min_1_day'), np.str_('temp'), np.str_('noise')]\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (autoregressive and exog) with SelectFromModel + SequentialFeatureSelector\n",
    "# ==============================================================================\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "\n",
    "# Step 1: Select the 70% most important features with SelectFromModel\n",
    "selector_1 = SelectFromModel(\n",
    "                 estimator    = estimator,\n",
    "                 max_features = int(data.shape[1] * 0.7),\n",
    "                 threshold    = -np.inf\n",
    "             )\n",
    "selected_lags_1, selected_window_features_1, selected_exog_1 = select_features(\n",
    "    forecaster  = forecaster,\n",
    "    selector    = selector_1,\n",
    "    y           = data[\"users\"],\n",
    "    exog        = data.drop(columns=\"users\"),\n",
    "    select_only = None,\n",
    "    subsample   = 0.2,\n",
    "    verbose     = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (SequentialFeatureSelector)\n",
      "---------------------------------------------------------\n",
      "Total number of records available: 8712\n",
      "Total number of records used for feature selection: 1742\n",
      "Number of features available: 62\n",
      "    Lags            (n=36)\n",
      "    Window features (n=1)\n",
      "    Exog            (n=25)\n",
      "Number of features selected: 25\n",
      "    Lags            (n=12) : [1, 6, 7, 8, 21, 24, 25, 28, 29, 30, 40, 47]\n",
      "    Window features (n=0) : []\n",
      "    Exog            (n=13) : ['hour_day_sin', 'hour_day_cos', 'poly_month_cos__week_of_year_sin', 'poly_week_day_sin__hour_day_sin', 'poly_week_day_sin__hour_day_cos', 'poly_week_day_sin__sunrise_hour_sin', 'poly_week_day_cos__hour_day_cos', 'poly_hour_day_sin__hour_day_cos', 'poly_hour_day_sin__sunrise_hour_cos', 'poly_hour_day_sin__sunset_hour_sin', 'poly_hour_day_cos__sunset_hour_sin', 'temp_roll_mean_7_day', 'temp']\n"
     ]
    }
   ],
   "source": [
    "# Step 2: Select the 25 most important features with SequentialFeatureSelector\n",
    "window_features_1 = RollingFeatures(stats=['mean'], window_sizes=24)\n",
    "forecaster.set_lags(lags=selected_lags_1)\n",
    "forecaster.set_window_features(window_features=window_features_1)\n",
    "\n",
    "selector_2 = SequentialFeatureSelector(\n",
    "                 estimator            = estimator,\n",
    "                 n_features_to_select = 25,\n",
    "                 direction            = \"forward\",\n",
    "                 cv                   = ShuffleSplit(n_splits=1, test_size=0.3, random_state=951),\n",
    "                 scoring              = \"neg_mean_absolute_error\",\n",
    "             )\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features(\n",
    "    forecaster  = forecaster,\n",
    "    selector    = selector_2,\n",
    "    y           = data[\"users\"],\n",
    "    exog        = data[selected_exog_1],\n",
    "    select_only = None,\n",
    "    subsample   = 0.2,\n",
    "    verbose     = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Selection in Global Forecasting Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with univariate forecasting models, feature selection can be applied to global forecasting models (multi-series). In this case, the `select_features_multiseries` function is used. This function has the same parameters as `select_features`, but the `y` parameter is replaced by `series`.\n",
    "\n",
    "- `forecaster`: Forecaster of type `ForecasterRecursiveMultiSeries` or `ForecasterDirectMultiVariate`.\n",
    "\n",
    "- `selector`: Feature selector from `sklearn.feature_selection`. For example, `RFE` or `RFECV`.\n",
    "\n",
    "- `series`: Target time series to which the feature selection will be applied.\n",
    "\n",
    "- `exog`: Exogenous variables.\n",
    "\n",
    "- `select_only`: Decide what type of features to include in the selection process. \n",
    "        \n",
    "    + If `'autoreg'`, only autoregressive features (lags or custom predictors) are evaluated by the selector. All exogenous features are included in the output `selected_exog`.\n",
    "\n",
    "    + If `'exog'`, only exogenous features are evaluated without the presence of autoregressive features. All autoregressive features are included in the outputs `selected_lags` and `selected_window_features`.\n",
    "\n",
    "    + If `None`, all features are evaluated by the selector.\n",
    "\n",
    "- `force_inclusion`: Features to force include in the final list of selected features.\n",
    "        \n",
    "    + If `list`, list of feature names to force include.\n",
    "    \n",
    "    + If `str`, regular expression to identify features to force include. For example, if `force_inclusion=\"^sun_\"`, all features that begin with \"sun_\" will be included in the final list of selected features.\n",
    "\n",
    "- `subsample`: Proportion of records to use for feature selection.\n",
    "\n",
    "- `random_state`: Sets a seed for the random subsample so that the subsampling process is always deterministic.\n",
    "\n",
    "- `verbose`: Print information about feature selection process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">╭─────────────────────── <span style=\"font-weight: bold\">items_sales</span> ───────────────────────╮\n",
       "│ <span style=\"font-weight: bold\">Description:</span>                                              │\n",
       "│ Simulated time series for the sales of 3 different items. │\n",
       "│                                                           │\n",
       "│ <span style=\"font-weight: bold\">Source:</span>                                                   │\n",
       "│ Simulated data.                                           │\n",
       "│                                                           │\n",
       "│ <span style=\"font-weight: bold\">URL:</span>                                                      │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-  │\n",
       "│ datasets/main/data/simulated_items_sales.csv              │\n",
       "│                                                           │\n",
       "│ <span style=\"font-weight: bold\">Shape:</span> 1097 rows x 3 columns                              │\n",
       "╰───────────────────────────────────────────────────────────╯\n",
       "</pre>\n"
      ],
      "text/plain": [
       "╭─────────────────────── \u001b[1mitems_sales\u001b[0m ───────────────────────╮\n",
       "│ \u001b[1mDescription:\u001b[0m                                              │\n",
       "│ Simulated time series for the sales of 3 different items. │\n",
       "│                                                           │\n",
       "│ \u001b[1mSource:\u001b[0m                                                   │\n",
       "│ Simulated data.                                           │\n",
       "│                                                           │\n",
       "│ \u001b[1mURL:\u001b[0m                                                      │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-  │\n",
       "│ datasets/main/data/simulated_items_sales.csv              │\n",
       "│                                                           │\n",
       "│ \u001b[1mShape:\u001b[0m 1097 rows x 3 columns                              │\n",
       "╰───────────────────────────────────────────────────────────╯\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Data\n",
    "# ==============================================================================\n",
    "data = fetch_dataset(name=\"items_sales\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_1</th>\n",
       "      <th>item_2</th>\n",
       "      <th>item_3</th>\n",
       "      <th>month</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>day_of_month</th>\n",
       "      <th>week_of_year</th>\n",
       "      <th>quarter</th>\n",
       "      <th>is_month_start</th>\n",
       "      <th>is_month_end</th>\n",
       "      <th>is_quarter_start</th>\n",
       "      <th>is_quarter_end</th>\n",
       "      <th>is_year_start</th>\n",
       "      <th>is_year_end</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-01-01</th>\n",
       "      <td>8.253175</td>\n",
       "      <td>21.047727</td>\n",
       "      <td>19.429739</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-02</th>\n",
       "      <td>22.777826</td>\n",
       "      <td>26.578125</td>\n",
       "      <td>28.009863</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-03</th>\n",
       "      <td>27.549099</td>\n",
       "      <td>31.751042</td>\n",
       "      <td>32.078922</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-04</th>\n",
       "      <td>25.895533</td>\n",
       "      <td>24.567708</td>\n",
       "      <td>27.252276</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-05</th>\n",
       "      <td>21.379238</td>\n",
       "      <td>18.191667</td>\n",
       "      <td>20.357737</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               item_1     item_2     item_3  month  day_of_week  day_of_month  \\\n",
       "date                                                                            \n",
       "2012-01-01   8.253175  21.047727  19.429739      1            6             1   \n",
       "2012-01-02  22.777826  26.578125  28.009863      1            0             2   \n",
       "2012-01-03  27.549099  31.751042  32.078922      1            1             3   \n",
       "2012-01-04  25.895533  24.567708  27.252276      1            2             4   \n",
       "2012-01-05  21.379238  18.191667  20.357737      1            3             5   \n",
       "\n",
       "            week_of_year  quarter  is_month_start  is_month_end  \\\n",
       "date                                                              \n",
       "2012-01-01            52        1               1             0   \n",
       "2012-01-02             1        1               0             0   \n",
       "2012-01-03             1        1               0             0   \n",
       "2012-01-04             1        1               0             0   \n",
       "2012-01-05             1        1               0             0   \n",
       "\n",
       "            is_quarter_start  is_quarter_end  is_year_start  is_year_end  \n",
       "date                                                                      \n",
       "2012-01-01                 1               0              1            0  \n",
       "2012-01-02                 0               0              0            0  \n",
       "2012-01-03                 0               0              0            0  \n",
       "2012-01-04                 0               0              0            0  \n",
       "2012-01-05                 0               0              0            0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create exogenous features based on the calendar\n",
    "# ==============================================================================\n",
    "data[\"month\"] = data.index.month\n",
    "data[\"day_of_week\"] = data.index.dayofweek\n",
    "data[\"day_of_month\"] = data.index.day\n",
    "data[\"week_of_year\"] = data.index.isocalendar().week\n",
    "data[\"quarter\"] = data.index.quarter\n",
    "data[\"is_month_start\"] = data.index.is_month_start.astype(int)\n",
    "data[\"is_month_end\"] = data.index.is_month_end.astype(int)\n",
    "data[\"is_quarter_start\"] = data.index.is_quarter_start.astype(int)\n",
    "data[\"is_quarter_end\"] = data.index.is_quarter_end.astype(int)\n",
    "data[\"is_year_start\"] = data.index.is_year_start.astype(int)\n",
    "data[\"is_year_end\"] = data.index.is_year_end.astype(int)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create forecaster\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursiveMultiSeries(\n",
    "    estimator       = LGBMRegressor(n_estimators=900, random_state=159, max_depth=7, verbose=-1),\n",
    "    lags            = 24,\n",
    "    window_features = RollingFeatures(stats=['mean', 'mean', 'mean'], window_sizes=[24, 48, 72])\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╭────────────────────────────────── InputTypeWarning ──────────────────────────────────╮</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Passing a DataFrame (either wide or long format) as `series` requires additional     <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> internal transformations, which can increase computational time. It is recommended   <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> to use a dictionary of pandas Series instead. For more details, see:                 <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> https://skforecast.org/latest/user_guides/independent-multi-time-series-forecasting. <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> html#input-data                                                                      <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>                                                                                      <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Category : skforecast.exceptions.InputTypeWarning                                    <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Location :                                                                           <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> tils.py:2349                                                                         <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Suppress : warnings.simplefilter('ignore', category=InputTypeWarning)                <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╰──────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[38;5;214m╭─\u001b[0m\u001b[38;5;214m─────────────────────────────────\u001b[0m\u001b[38;5;214m InputTypeWarning \u001b[0m\u001b[38;5;214m─────────────────────────────────\u001b[0m\u001b[38;5;214m─╮\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Passing a DataFrame (either wide or long format) as `series` requires additional     \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m internal transformations, which can increase computational time. It is recommended   \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m to use a dictionary of pandas Series instead. For more details, see:                 \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m https://skforecast.org/latest/user_guides/independent-multi-time-series-forecasting. \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m html#input-data                                                                      \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m                                                                                      \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Category : skforecast.exceptions.InputTypeWarning                                    \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Location :                                                                           \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m tils.py:2349                                                                         \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Suppress : warnings.simplefilter('ignore', category=InputTypeWarning)                \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m╰──────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╭────────────────────────────────── DataTypeWarning ───────────────────────────────────╮</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> `exog` may contain only `int`, `float` or `category` dtypes. Most machine learning   <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> models do not allow other types of values. Fitting the forecaster may fail.          <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>                                                                                      <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Category : skforecast.exceptions.DataTypeWarning                                     <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Location :                                                                           <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> tils.py:640                                                                          <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Suppress : warnings.simplefilter('ignore', category=DataTypeWarning)                 <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╰──────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[38;5;214m╭─\u001b[0m\u001b[38;5;214m─────────────────────────────────\u001b[0m\u001b[38;5;214m DataTypeWarning \u001b[0m\u001b[38;5;214m──────────────────────────────────\u001b[0m\u001b[38;5;214m─╮\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m `exog` may contain only `int`, `float` or `category` dtypes. Most machine learning   \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m models do not allow other types of values. Fitting the forecaster may fail.          \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m                                                                                      \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Category : skforecast.exceptions.DataTypeWarning                                     \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Location :                                                                           \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m tils.py:640                                                                          \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Suppress : warnings.simplefilter('ignore', category=DataTypeWarning)                 \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m╰──────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive feature elimination (RFECV)\n",
      "-------------------------------------\n",
      "Total number of records available: 3075\n",
      "Total number of records used for feature selection: 1537\n",
      "Number of features available: 38\n",
      "    Lags            (n=24)\n",
      "    Window features (n=3)\n",
      "    Exog            (n=11)\n",
      "Number of features selected: 28\n",
      "    Lags            (n=24) : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]\n",
      "    Window features (n=2) : ['roll_mean_24', 'roll_mean_72']\n",
      "    Exog            (n=2) : ['day_of_week', 'week_of_year']\n"
     ]
    }
   ],
   "source": [
    "# Feature selection (autoregressive and exog) with scikit-learn RFECV\n",
    "# ==============================================================================\n",
    "series_columns = [\"item_1\", \"item_2\", \"item_3\"]\n",
    "exog_columns = [col for col in data.columns if col not in series_columns]\n",
    "estimator = LGBMRegressor(n_estimators=100, max_depth=5, random_state=15926, verbose=-1)\n",
    "selector = RFECV(estimator=estimator, step=1, cv=3, min_features_to_select=25)\n",
    "\n",
    "selected_lags, selected_window_features, selected_exog = select_features_multiseries(\n",
    "    forecaster      = forecaster,\n",
    "    selector        = selector,\n",
    "    series          = data[series_columns],\n",
    "    exog            = data[exog_columns],\n",
    "    select_only     = None,\n",
    "    force_inclusion = None,\n",
    "    subsample       = 0.5,\n",
    "    random_state    = 123,\n",
    "    verbose         = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the best subset of features has been selected, the global forecasting model is trained with the selected features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╭────────────────────────────────── InputTypeWarning ──────────────────────────────────╮</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Passing a DataFrame (either wide or long format) as `series` requires additional     <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> internal transformations, which can increase computational time. It is recommended   <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> to use a dictionary of pandas Series instead. For more details, see:                 <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> https://skforecast.org/latest/user_guides/independent-multi-time-series-forecasting. <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> html#input-data                                                                      <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>                                                                                      <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Category : skforecast.exceptions.InputTypeWarning                                    <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Location :                                                                           <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> tils.py:2349                                                                         <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Suppress : warnings.simplefilter('ignore', category=InputTypeWarning)                <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╰──────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[38;5;214m╭─\u001b[0m\u001b[38;5;214m─────────────────────────────────\u001b[0m\u001b[38;5;214m InputTypeWarning \u001b[0m\u001b[38;5;214m─────────────────────────────────\u001b[0m\u001b[38;5;214m─╮\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Passing a DataFrame (either wide or long format) as `series` requires additional     \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m internal transformations, which can increase computational time. It is recommended   \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m to use a dictionary of pandas Series instead. For more details, see:                 \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m https://skforecast.org/latest/user_guides/independent-multi-time-series-forecasting. \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m html#input-data                                                                      \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m                                                                                      \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Category : skforecast.exceptions.InputTypeWarning                                    \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Location :                                                                           \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m tils.py:2349                                                                         \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Suppress : warnings.simplefilter('ignore', category=InputTypeWarning)                \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m╰──────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╭────────────────────────────────── DataTypeWarning ───────────────────────────────────╮</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> `exog` may contain only `int`, `float` or `category` dtypes. Most machine learning   <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> models do not allow other types of values. Fitting the forecaster may fail.          <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>                                                                                      <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Category : skforecast.exceptions.DataTypeWarning                                     <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Location :                                                                           <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> tils.py:640                                                                          <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span> Suppress : warnings.simplefilter('ignore', category=DataTypeWarning)                 <span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">│</span>\n",
       "<span style=\"color: #ffaf00; text-decoration-color: #ffaf00\">╰──────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[38;5;214m╭─\u001b[0m\u001b[38;5;214m─────────────────────────────────\u001b[0m\u001b[38;5;214m DataTypeWarning \u001b[0m\u001b[38;5;214m──────────────────────────────────\u001b[0m\u001b[38;5;214m─╮\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m `exog` may contain only `int`, `float` or `category` dtypes. Most machine learning   \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m models do not allow other types of values. Fitting the forecaster may fail.          \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m                                                                                      \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Category : skforecast.exceptions.DataTypeWarning                                     \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Location :                                                                           \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m c:\\Users\\jaesc2\\Miniconda3\\envs\\skforecast_py12\\Lib\\site-packages\\skforecast\\utils\\u \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m tils.py:640                                                                          \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m│\u001b[0m Suppress : warnings.simplefilter('ignore', category=DataTypeWarning)                 \u001b[38;5;214m│\u001b[0m\n",
       "\u001b[38;5;214m╰──────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 {\n",
       "            font-family: 'Arial', sans-serif;\n",
       "            font-size: 0.9em;\n",
       "            color: #333333;\n",
       "            border: 1px solid #ddd;\n",
       "            background-color: #f0f8ff;\n",
       "            padding: 5px 15px;\n",
       "            border-radius: 8px;\n",
       "            max-width: 600px;\n",
       "            #margin: auto;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 h2 {\n",
       "            font-size: 1.5em;\n",
       "            color: #222222;\n",
       "            border-bottom: 2px solid #ddd;\n",
       "            padding-bottom: 5px;\n",
       "            margin-bottom: 15px;\n",
       "            margin-top: 5px;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 details {\n",
       "            margin: 10px 0;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 summary {\n",
       "            font-weight: bold;\n",
       "            font-size: 1.1em;\n",
       "            color: #000000;\n",
       "            cursor: pointer;\n",
       "            margin-bottom: 5px;\n",
       "            background-color: #b3dbfd;\n",
       "            padding: 5px;\n",
       "            border-radius: 5px;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 summary:hover {\n",
       "            color: #000000;\n",
       "            background-color: #e0e0e0;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 ul {\n",
       "            font-family: 'Courier New', monospace;\n",
       "            list-style-type: none;\n",
       "            padding-left: 20px;\n",
       "            margin: 10px 0;\n",
       "            line-height: normal;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 li {\n",
       "            margin: 5px 0;\n",
       "            font-family: 'Courier New', monospace;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 li strong {\n",
       "            font-weight: bold;\n",
       "            color: #444444;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 li::before {\n",
       "            content: \"- \";\n",
       "            color: #666666;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 a {\n",
       "            color: #001633;\n",
       "            text-decoration: none;\n",
       "        }\n",
       "        .container-8d7cb86e0ba94edbb1570c80c0495053 a:hover {\n",
       "            color: #359ccb; \n",
       "        }\n",
       "    </style>\n",
       "    \n",
       "        <div class=\"container-8d7cb86e0ba94edbb1570c80c0495053\">\n",
       "            <p style=\"font-size: 1.5em; font-weight: bold; margin-block-start: 0.83em; margin-block-end: 0.83em;\">ForecasterRecursiveMultiSeries</p>\n",
       "            <details open>\n",
       "                <summary>General Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Estimator:</strong> LGBMRegressor</li>\n",
       "                    <li><strong>Lags:</strong> [ 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]</li>\n",
       "                    <li><strong>Window features:</strong> ['roll_mean_24', 'roll_mean_72']</li>\n",
       "                    <li><strong>Window size:</strong> 72</li>\n",
       "                    <li><strong>Series encoding:</strong> ordinal</li>\n",
       "                    <li><strong>Exogenous included:</strong> True</li>\n",
       "                    <li><strong>Weight function included:</strong> False</li>\n",
       "                    <li><strong>Series weights:</strong> None</li>\n",
       "                    <li><strong>Differentiation order:</strong> None</li>\n",
       "                    <li><strong>Creation date:</strong> 2025-11-26 15:35:55</li>\n",
       "                    <li><strong>Last fit date:</strong> 2025-11-26 15:35:57</li>\n",
       "                    <li><strong>Skforecast version:</strong> 0.19.0</li>\n",
       "                    <li><strong>Python version:</strong> 3.12.11</li>\n",
       "                    <li><strong>Forecaster id:</strong> None</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Exogenous Variables</summary>\n",
       "                <ul>\n",
       "                    day_of_week, week_of_year\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Data Transformations</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Transformer for series:</strong> None</li>\n",
       "                    <li><strong>Transformer for exog:</strong> None</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Training Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Series names (levels):</strong> item_1, item_2, item_3</li>\n",
       "                    <li><strong>Training range:</strong> 'item_1': ['2012-01-01', '2015-01-01'], 'item_2': ['2012-01-01', '2015-01-01'], 'item_3': ['2012-01-01', '2015-01-01']</li>\n",
       "                    <li><strong>Training index type:</strong> DatetimeIndex</li>\n",
       "                    <li><strong>Training index frequency:</strong> <Day></li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Estimator Parameters</summary>\n",
       "                <ul>\n",
       "                    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0, 'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': 7, 'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0, 'n_estimators': 900, 'n_jobs': None, 'num_leaves': 31, 'objective': None, 'random_state': 159, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0, 'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Fit Kwargs</summary>\n",
       "                <ul>\n",
       "                    {}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <p>\n",
       "                <a href=\"https://skforecast.org/0.19.0/api/forecasterrecursivemultiseries.html\">&#128712 <strong>API Reference</strong></a>\n",
       "                &nbsp;&nbsp;\n",
       "                <a href=\"https://skforecast.org/0.19.0/user_guides/independent-multi-time-series-forecasting.html\">&#128462 <strong>User Guide</strong></a>\n",
       "            </p>\n",
       "        </div>\n",
       "        "
      ],
      "text/plain": [
       "============================== \n",
       "ForecasterRecursiveMultiSeries \n",
       "============================== \n",
       "Estimator: LGBMRegressor \n",
       "Lags: [ 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24] \n",
       "Window features: ['roll_mean_24', 'roll_mean_72'] \n",
       "Window size: 72 \n",
       "Series encoding: ordinal \n",
       "Series names (levels): item_1, item_2, item_3 \n",
       "Exogenous included: True \n",
       "Exogenous names: day_of_week, week_of_year \n",
       "Transformer for series: None \n",
       "Transformer for exog: None \n",
       "Weight function included: False \n",
       "Series weights: None \n",
       "Differentiation order: None \n",
       "Training range: \n",
       "    'item_1': ['2012-01-01', '2015-01-01'], 'item_2': ['2012-01-01', '2015-01-01'],\n",
       "    'item_3': ['2012-01-01', '2015-01-01'] \n",
       "Training index type: DatetimeIndex \n",
       "Training index frequency: <Day> \n",
       "Estimator parameters: \n",
       "    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0,\n",
       "    'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': 7,\n",
       "    'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0,\n",
       "    'n_estimators': 900, 'n_jobs': None, 'num_leaves': 31, 'objective': None,\n",
       "    'random_state': 159, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0,\n",
       "    'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1} \n",
       "fit_kwargs: {} \n",
       "Creation date: 2025-11-26 15:35:55 \n",
       "Last fit date: 2025-11-26 15:35:57 \n",
       "Skforecast version: 0.19.0 \n",
       "Python version: 3.12.11 \n",
       "Forecaster id: None "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train forecaster with selected features\n",
    "# ==============================================================================\n",
    "new_window_features = RollingFeatures(stats=['mean', 'mean'], window_sizes=[24, 72])\n",
    "forecaster.set_lags(lags=selected_lags)\n",
    "forecaster.set_window_features(window_features=new_window_features)\n",
    "\n",
    "forecaster.fit(series=data[series_columns], exog=data[selected_exog])\n",
    "forecaster"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "skforecast_py12",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Tabla de contenidos",
   "title_sidebar": "Tabla de contenidos",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "position": {
    "height": "144.391px",
    "left": "1478px",
    "right": "20px",
    "top": "126px",
    "width": "350px"
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
